Method and apparatus for an improved localizer for 3D imaging

ABSTRACT

This patent includes a method for performing precision localization within the body on a cross-sectional imaging examination. As a result of the improved precision localization techniques, anatomic structure specific coordinate systems can be developed to improve radiological analysis.

CROSS-REFERENCES TO RELATED APPLICATIONS

This patent application is a continuation-in-part of U.S. patentapplication Ser. No. 16/785,606 filed on Feb. 9, 2020 and claims thebenefit of U.S. Provisional Application 62/939,685 filed on Nov. 25,2019 and U.S. Provisional Application 62/940,822 filed on Nov. 26, 2019.

TECHNICAL FIELD

This patent application applies to the field of 3D imaging.

BACKGROUND

The field of 3D visualization is growing rapidly in medicine, military,video games and many other industries.

SUMMARY

All examples, aspects and features mentioned in this document can becombined in any technically possible way.

Methods disclosed in the patent are provided to overcome some of thedifficulties that are faced during localization of 3D imaging of complexdatasets.

A first difficulty is discussed in medical imaging. A radiologist maydescribe using language where a lesion is located. For example, asentence may state “an indeterminate T2 hyperintensity in the rightfrontal lobe.” But, if the radiologist just said that, another physicianmay be left with the question of “where in the right frontal lobe.”Assume the radiologist tried harder and said “an indeterminate T2hyperintensity in the superior frontal gyms of the right frontal lobe.”Still, a reader of the radiology report could ask “where precisely inthe superior frontal gyms of the right frontal lobe?” For this reason,radiologist sometimes leave an image number along side of thedescription of their finding, yet a number can be inadequate. Forexample, the radiologist states “in the superior frontal gyms of theright frontal lobe (series 4: image 120)”. Another radiologist andreferring clinician might still be left wondering what exactly was thatradiologist referring to?

A second difficulty is discussed in medical imaging. Currentlocalization strategies utilize a matrix (e.g., 512×512) and a series ofCT slices. When a user performs localization from a CT scan at a firsttime point to a CT scan at a second time point, the localizer performslocalization based on slice number and pixel coordinate. Since apatient's anatomy comprises soft tissues that are deformable and sincethe patient may be angled or positioned slightly different on twodifferent examinations, the anatomy does not perfectly align. Forexample, if a user using a current PACS localization process performsslice sync, scrolls to a spot and subsequently performs localization ofthe right adrenal gland from a 2019 examination to a 2020 examination,the localization digital object localizes to a slice and a pixelcoordinate, but the slice may not even include the adrenal gland andfurthermore even if the slice did include the adrenal gland, thelocalizer would not necessarily be positioned on the right adrenal glandin the 2020 examination. The methods and apparatuses disclosed in thispatent provide an improved localization. The methods disclosed provide aGPS like system for the human body.

The preferred embodiment loads a first 3D imaging dataset into an imageprocessing workstation wherein the first 3D dataset comprises avoxelated dataset of a scanned volume at a first time point.Additionally, the preferred embodiment loads a second 3D imaging datasetinto the image processing workstation wherein the second 3D datasetcomprises a voxelated dataset of the scanned volume at a second timepoint. Additionally, the preferred embodiment performs segmentation ofthe first 3D imaging dataset to define a structure. Additionally, thepreferred embodiment performs segmentation of the second 3D imagingdataset to define the structure. Additionally, the preferred embodimentperforms a smart localization system comprising: positioning a cursor ona first coordinate of an image of the first 3D dataset, wherein thefirst coordinate is enclosed within the structure, and wherein the firstcoordinate is located at a sub-structure location. A sub-structurelocation is a location inside the structure. Additionally, the preferredembodiment determines a corresponding first coordinate in the second 3Ddataset, wherein the corresponding first coordinate is enclosed withinthe structure, wherein the corresponding first coordinate is located atthe sub-structure location. Additionally, the preferred embodimentdisplays a digital object at the corresponding first coordinate in animage of the second 3D dataset. An alternative embodiment is to displayan imaging slice of the second 3D dataset containing the sub-structure.

In order to accomplish this, the organ must be segmented such that theboundaries of the organ are properly defined. For example, in theabdomen, each solid organ (e.g., liver, spleen, gallbladder, pancreas,etc.) would be segmented such that its margins are well-defined and theinner aspects of the organ are contained in a volume. For example, theliver capsule would be the outer margin of the liver and the liverparenchyma would be in the inside of the volume.

Additionally, the coordinates of the outer margin of the organ may aidin this method's precision localization. For example, the coordinates atthe top slice of the organ that demarcate the outer margins of the organbased on the segmentation method applied above should be recorded. Forexample, for a particular axial slice (z-value is fixed), the each (x,y)coordinate at the perimeter of the organ should be recorded and matchedwith the z-coordinate of the axial slice. This process is repeated foreach axial slice until at which point all (x,y,z) coordinates at theboundary of the organ are recorded. Note that this reference point has atraditional (x,y,z) coordinate in each examination, but will serve to bethe critical, reproducible reference points for the boundaries of theorgan for the organ-specific coordinate system which will serve toregister the organ over multiple examinations.

Note that the described reference points have traditional (x,y,z)coordinates in each examination, but will serve to be the critical,reproducible reference points for the organ-specific coordinate systemwhich will serve to register the organ center over multipleexaminations.

Some embodiments comprise utilizing in the smart localization system atleast one reference point within the structure comprising wherein the atleast one reference point is selected from the group consisting of: acenter point; a superior most point; an inferior most point; a medialmost point; a lateral most point; an anterior most point; a posteriormost point; and a recognizable anatomic feature.

Alternative embodiments would be to select the superior most voxel,anterior most voxel or other algorithms could be used to achievereproducible reference points. In organs wherein the internal parenchymademonstrates recognizable features, the internal sub-structures canserve as precision landmarks and registration point(s). For example, inthe liver the internal sub-structure of the middle hepatic vein can beused. For example, in the brain, the center point of the caudate headcan be used.

Some embodiments comprise wherein the at least one reference point isused for at least one of the group consisting of: volumetric analysis;and, morphologic analysis.

Some embodiments comprise utilizing at least one pseudoreference pointwithin the structure comprising wherein the at least one pseudoreferencepoint is a distance in between at least two reference points.

Some embodiments comprise wherein the at least one pseudoreference pointis used for at least one of the group consisting of: volumetricanalysis; and, morphologic analysis.

Some embodiments comprise utilizing in the smart localization system acoordinate system for the structure based comprising at least one of thegroup consisting of: a cartesian coordinate system; a cylindricalcoordinate system; a polar coordinate system; a spherical coordinatesystem; and an organ specific coordinate system.

Some embodiments comprise assigning a precision location of a lesionwherein the precision location comprises a coordinate location on thecoordinate system.

Some embodiments comprise inputting the precision location of the lesionin a radiology report.

Some embodiments comprise inputting an annotation at the site of theprecision location of the lesion on an image.

Some embodiments comprise generating at least two coordinate systems forthe structure.

Some embodiments comprise using the coordinate system for at least oneof the group consisting of: a radiation treatment; and a surgicalprocedure.

Some embodiments comprise wherein when the structure changes in sizefrom the first 3D dataset to the second 3D dataset, the determining ofthe corresponding first coordinate in the second 3D dataset accounts forthe structure's changes in size.

Some embodiments comprise wherein when the structure changes inconfiguration from the first 3D dataset to the second 3D dataset, thedetermining of the corresponding first coordinate in the second 3Ddataset accounts for the structure's changes in configuration.

Some embodiments comprise wherein when the structure changes inorientation from the first 3D dataset to the second 3D dataset, thedetermining of the corresponding first coordinate in the second 3Ddataset accounts for the structure's changes in orientation.

Some embodiments comprise determining a reference axis for the volume inthe first 3D dataset; determining an axis of the structure in the first3D dataset; determining a first angle wherein the first angle is anangle between the reference axis for the volume in the first 3D datasetand the axis of the structure in the first 3D dataset; determining acorresponding reference axis for the volume in the second 3D dataset;determining a corresponding axis of the structure in the second 3Ddataset; determining a second angle wherein the second angle is an anglebetween the corresponding reference axis for the volume in the second 3Ddataset and the corresponding axis of the structure in the second 3Ddataset; and comparing the first angle with the second angle todetermine an interval change.

Some embodiments comprise performing an analysis of interval changebetween a voxel at the sub-structure in the first 3D dataset and a voxelat the sub-structure in the second 3D dataset.

Some embodiments comprise determining the corresponding first coordinateby utilizing an artificial intelligence system, wherein the artificialintelligence system utilizes training data comprises sets oflongitudinal 3D imaging examinations with embedded localization points.

Some embodiments comprise a non-transitory computer readable mediumhaving computer readable code thereon for image processing, the mediumcomprising: performing segmentation of a first 3D imaging dataset todefine a structure wherein the first 3D dataset comprises a voxelateddataset of a scanned volume at a first time point; performingsegmentation of a second 3D imaging dataset to define the structurewherein the second 3D dataset comprises a voxelated dataset of thescanned volume at a second time point; performing a smart localizationsystem comprising: positioning a cursor on a first coordinate of animage of the first 3D dataset, wherein the first coordinate is enclosedwithin the structure, and wherein the first coordinate is located at asub-structure location; and determining a corresponding first coordinatein the second 3D dataset, wherein the corresponding first coordinate isenclosed within the structure, wherein the corresponding firstcoordinate is located at the sub-structure location; and displaying adigital object at the corresponding first coordinate in an image of thesecond 3D dataset.

This method teaches a process to develop an organ specific coordinatesystem and a method to precision localization of which anatomic featureon a first examination corresponds to which anatomic feature on asubsequent examination. Specifically, by implementing the methodsdisclosed in this patent, the user will be able to click a mouse on aparticular punctate structure on a prior examination and then alocalizer pop-up will appear on the adjacent monitor showing the preciselocation of the localizer on the subsequent examination. For example,the user would be able to click on the right lateral most tip of the L4transverse process and processes disclosed in this patent will enable apop-up to appear on the subsequent examination precisely at the rightlateral most tip of the L4 transverse process. This example was toconvey in the most straight forward process the overall goal of thesystem. In practice, however, the radiologist might click on a new tinyhypodense lesion in the liver on a CT scan and wonders whether it waspresent on the prior examinations? The radiologist will be able to clickon the hypodense lesion and by applying methods disclosed in thispatent, the system will show a pop-up icon on the prior examination atthe exact corresponding spot and the radiologist will be able to tell inan instant whether the hypodense lesion is new or not. The saves time byreducing scrolling. Current systems are only able to link slices; thus,this algorithm will improve the radiologist's ability to track lesionsover time. In the preferred embodiment, this process of organ specificcoordinate system would be performed for all organs in the body.

It should be noted that some organs are relatively fixed in location(e.g., kidney since it is relatively fixed in the retroperitoneum). Notethat other organs are relatively mobile in location (e.g., liver whichis somewhat more mobile relating to diaphragmatic motion). Techniques ofthe smart localization system disclosed herein provide accuratelocalization of both fixed and mobile organs.

In some embodiments, at least one, but preferably multiple referencepoint within the organ may be established. Reference point(s) can beidentifiable and reproducible, so that the organ specific coordinatesystem can be established repeatedly over multiple examinations, evenwhen the solid organ is in different configurations or positions. Thereference point may be determined by human (e.g., radiologist) or by acomputer through a software (e.g., Artificial Intelligence) program.

In some embodiments, the center point of an organ is used as a referencepoint. The preferred embodiment would be to, after the segmentation hasbeen performed, assign the center point of the organ by the followingprocess. Consider a voxelated 3D dataset of a CT Scan wherein each voxelhas a data unit and an x, y, z coordinate. Record the maximum x-valueand minimum x-value of the organ. Then make the x-coordinate of thecenter, called centerX, of the organ at the half-way point between themaximum x-value and minimum x-value of the organ. Next, record themaximum y-value and minimum y-value of the organ. Then make they-coordinate of the center, called centerY, of the organ at the half-waypoint between the maximum y-value and minimum y-value of the organ.Record the maximum z-value and minimum z-value of the organ. Then makethe z-coordinate of the center, called centerZ, of the organ at thehalf-way point between the maximum Z-value and minimum Z-value of theorgan. Thus, using this preferred embodiment, the center of the organwould be located at (centerX, centerY, centerZ).

In some embodiments, points other than the center should be used as areference point. For example, consider the breast. Specifically, thepreferred embodiment for the reference point for the breast is thenipple, since physicians are already accustomed to using this as areference point. Another embodiment is to select reference points couldbe an imaging feature inside the organ, which is easily recognizable(e.g., surgical clip). Another embodiment is to select reference pointscorresponding to a reproducibly placed skin marker (e.g., BB marker overa birthmark).

Another embodiment would be to select reference points with an organ inclose proximity to a fixed (i.e., non-mobile) structure. For example,the bladder is securely fixed inferiorly to the prostate gland, so usingthis embodiment, a reference point near the base of the bladder could beperformed.

Some embodiments use an adaptable smart localization system as describedabove with traditional methods. For example, if a 2019 examinationincludes the right kidney and a 2020 examination does not (e.g., thepatient has had a right nephrectomy), the smart localization system cansignal to the user (e.g., via pop up) that the right kidney is no longerpresent. A localizer could be programmed to be placed in the right renalfossa, which would give the user context.

It should be noted that these techniques can be applied to non-medicalapplications, such examining 3D datasets for change, such as monitoringglaciers or ice burgs for purposes of climate change. A wide range ofnon-medical applications are also possible, such as those discussed inU.S. Provisional 62/940,822, which is incorporated by reference in itsentirety.

Still other embodiments include a computerized device, configured toprocess all the method operations disclosed herein as embodiments of theinvention. In such embodiments, the computerized device includes amemory system, a processor, communications interface in aninterconnection mechanism connecting these components. The memory systemis encoded with a process that provides steps explained herein that whenperformed (e.g., when executing) on the processor, operates as explainedherein within the computerized device to perform all of the methodembodiments and operations explained herein as embodiments of theinvention. Thus, any computerized device that performs or is programmedto perform processing explained herein is an embodiment of theinvention.

Other arrangements of embodiments of the invention that are disclosedherein include software programs to perform the method embodiment stepsand operations summarized above and disclosed in detail below. Moreparticularly, a computer program product is one embodiment that has acomputer-readable medium including computer program logic encodedthereon that when performed in a computerized device provides associatedoperations providing steps as explained herein.

The computer program logic, when executed on at least one processor witha computing system, causes the processor to perform the operations(e.g., the methods) indicated herein as embodiments of the invention.Such arrangements of the invention are typically provided as software,code and/or other data structures arranged or encoded on a computerreadable medium such as an optical medium (e.g., CD-ROM), floppy or harddisk or other a medium such as firmware or microcode in one or more ROMor RAM or PROM chips or as an Application Specific Integrated Circuit(ASIC) or as downloadable software images in one or more modules, sharedlibraries, etc. The software or firmware or other Such configurationscan be installed onto a computerized device to cause one or moreprocessors in the computerized device to perform the techniquesexplained herein as embodiments of the invention. Software processesthat operate in a collection of computerized devices, such as in a groupof data communications devices or other entities can also provide thesystem of the invention. The system of the invention can be distributedbetween many software processes on several data communications devices,or all processes could run on a small set of dedicated computers, or onone computer alone.

It is to be understood that the embodiments of the invention can beembodied strictly as a software program, as software and hardware, or ashardware and/or circuitry alone. Such as within a data communicationsdevice. The features of the invention, as explained herein, may beemployed in data processing devices and/or software systems for suchdevices. Note that each of the different features, techniques,configurations, etc. discussed in this disclosure can be executedindependently or in combination. Accordingly, the present invention canbe embodied and viewed in many different ways. Also, note that thisSummary section herein does not specify every embodiment and/orincrementally novel aspect of the present disclosure or claimedinvention. Instead, this summary only provides a preliminary discussionof different embodiments and corresponding points of novelty overconventional techniques. For additional details, elements, and/orpossible perspectives (permutations) of the invention, the reader isdirected to the Detailed Description section and corresponding figuresof the present disclosure as further discussed below.

BRIEF DESCRIPTION OF FIGURES

The flow diagrams do not depict the syntax of any particular programminglanguage. Rather, the flow diagrams illustrate the functionalinformation one of ordinary skill in the art requires to fabricatecircuits or to generate computer software to perform the processingrequired in accordance with the present invention. It should be notedthat many routine program elements, such as initialization of loops andvariables and the use of temporary variables, are not shown. It will beappreciated by those of ordinary skill in the art that unless otherwiseindicated herein, the particular sequence of steps described isillustrative only and can be varied without departing from the spirit ofthe invention. Thus, unless otherwise stated the steps described beloware unordered meaning that, when possible, the steps can be performed inany convenient or desirable order.

The foregoing will be apparent from the following more particulardescription of preferred embodiments of the invention, as illustrated inthe accompanying drawings in which like reference characters refer tothe same parts throughout the different views. The drawings are notnecessarily to scale, emphasis instead being placed upon illustratingthe principles of the invention.

FIG. 1 illustrates prior art showing the current coordinate system for across-sectional imaging examination.

FIG. 2A illustrates the prior art system of localizing.

FIG. 2B illustrates scrolling with a localizing link.

FIG. 3 illustrates a radiologist's process for re-aligning the twodifferent CT examinations.

FIG. 4 illustrates the smart localization system.

FIG. 5 illustrates techniques to determine the corresponding firstcoordinate.

FIG. 6A illustrates a first example of the smart localization systemwherein the first coordinate is determined by a cursor.

FIG. 6B illustrates a first example of the smart localization systemwherein the first coordinate is determined by a fixation location.

FIG. 7 illustrates display options for the corresponding firstcoordinate.

FIG. 8 illustrates improvements, which are enabled by implementing anorgan-specific coordinate system.

FIG. 9A illustrates identification of internal reference points at aninitial examination and two spots where the user wants to localize to ona subsequent examination.

FIG. 9B illustrates identification of internal reference points at asubsequent examination with two spots where the user has localized to ascompared to the initial examination.

FIG. 10A illustrates the liver and an example anatomic structurespecific coordinate system.

FIG. 10B illustrates a CT image of the liver and example coordinate of avoxel in the liver.

FIG. 11 illustrates the adrenal gland and two different organ specificcoordinate systems.

FIG. 12 illustrates reference points, pseudoreference points and ensuinganalysis.

FIG. 13 illustrates the plotting of organ specific voxel coordinates andtheir associated data units.

FIG. 14A illustrates a specific lesion in an organ.

FIG. 14B illustrates how a coordinate system can track a lesion in agrowing organ.

FIG. 15A illustrates a specific lesion in an organ, wherein the organ isin a first orientation.

FIG. 15B illustrates how an organ specific coordinate system can track alesion in a subsequent scan wherein the organ has a differentorientation.

FIG. 16A illustrates a specific lesion in an organ, wherein the organ isin a first configuration.

FIG. 16B illustrates how an organ specific coordinate system can track alesion in a subsequent scan wherein the organ has a differentconfiguration.

FIG. 17 illustrates multiple coordinate systems in a single structure.

FIG. 18 illustrates the use of an anatomic structure-specific coordinatesystem for each segmented structure in an imaging examination.

FIG. 19A illustrates a single coordinate system for the liver.

FIG. 19B illustrates a multiple coordinate systems for the liver.

DETAILED DESCRIPTION OF FIGURES

Some aspects, features and implementations described herein may includemachines such as computers, electronic components, optical components,and processes such as computer-implemented steps. It will be apparent tothose of ordinary skill in the art that the computer-implemented stepsmay be stored as computer-executable instructions on a non-transitorycomputer-readable medium. Furthermore, it will be understood by those ofordinary skill in the art that the computer-executable instructions maybe executed on a variety of tangible processor devices. For ease ofexposition, not every step, device or component that may be part of acomputer or data storage system is described herein. Those of ordinaryskill in the art will recognize such steps, devices and components inview of the teachings of the present disclosure and the knowledgegenerally available to those of ordinary skill in the art. Thecorresponding machines and processes are therefore enabled and withinthe scope of the disclosure.

FIG. 1 illustrates prior art showing the current coordinate system for across-sectional imaging examination. Currently in 2019, typicalradiology picture archiving communication systems (PACS) systemscommonly display the (x,y) value and the slice number (relates toz-coordinate). There is no value whatsoever for the radiologist to lookup at the screen and study the (x,y) values for a given slice. Theradiologist cares about what he/she sees on the screen specifically theanatomic structures and associated intensity unit, but does not careabout (x,y) coordinate. If the (x,y) coordinates were shifted a few mmaway, it would have no consequence. In other words, the diagnosticradiologist studies the anatomy and grayscales and searches forpathology. When a radiologist sees a finding, he measures the grayscale(e.g., intensity units) and looks at the margins, shape, etc. But, thereis no interpretation value whatsoever where the imaging finding fallswithin the 512×512 matrix. This image was displayed using Osirixsoftware suite, which uses axes as displayed above.

FIG. 2A illustrates the prior art system of localizing. Localization isused by radiologists to improve comparison between scans from twodifferent time points. To accomplish this, the user scrolls to a sliceon a first examination. Next, the radiologist scrolls to the similarslice on the second examination. In this example, a radiologist hasperformed the “link” command at the level of the ischial tuberosity onboth the left image (from 2019) and the right image (from 2018). Next,the radiologist performs the “link” command. Next the radiologistscrolls on the first examination and the second examination moves alongwith it, an example of which is shown in FIG. 2B below.

FIG. 2B illustrates scrolling with a localizing link. In this example, aradiologist has scrolled superiorly from the level of the ischialtuberosity, which was performed on the 2019 imaging examination. In the2019 exam, scrolling through 56 slices (as compared to FIG. 2A) hasoccurred and the position moved from position −1034 to position −1314,which is equal to 280 mm (56 slices, each slice is 5 mm thick). See textfrom the image above. In the 2016 exam, a distance of 280 mm (56 slices,each slice is 5 mm thick) have also been traversed. It is important tonote that at this new level, the adrenal glands are not lined up.Specifically, the left adrenal gland appears lambda shaped in 2019 andlinear in 2020. The radiologist has to perform manual override byunlinking and scrolling on only one of the examinations to re-align thestudies.

FIG. 3 illustrates a radiologist's process for re-aligning the twodifferent CT examinations. A radiologist needs to “unlink” the twoimaging examinations. Then, manually scroll on one of the imagingexaminations. In this example, the radiologist scrolled two additionalslices (58 in total) from the ischial tuberosity on the 2019examination. After doing so, the left adrenal glands became aligned. Theprocess of linking and unlinking can be performed to scroll throughsimilar slices during viewing.

FIG. 4 illustrates the smart localization system. 400 illustrates aprocessing block of loading a first 3D imaging dataset into an imageprocessing workstation wherein the first 3D dataset comprises avoxelated dataset of a scanned volume at a first time point. An optionis to perform segmentation of the first 3D imaging dataset at the firsttime point. This is useful by helping the smart localization systemunderstand where exactly the user is looking (e.g., which structure).401 illustrates a processing block of loading a second 3D imagingdataset into the image processing workstation wherein the second 3Ddataset comprises a voxelated dataset of the scanned volume at a secondtime point. The second 3D imaging dataset can also be segmented, forreasons discussed below. 402 illustrates a processing block ofperforming a smart localization system comprising: determining a firstcoordinate of an image of a first 3D dataset wherein the firstcoordinate is enclosed within a structure wherein the first coordinateis located at a sub-structure location by at least one of the groupconsisting of: positioning a cursor; and utilizing an eye trackingsystem; and determining a corresponding first coordinate in the second3D dataset, wherein the corresponding first coordinate is enclosedwithin the structure, wherein the corresponding first coordinate islocated at the sub-structure location. A matching system is usefulherein. For example, if a segmentation algorithm is applied to the firstimage and a cursor is determined to be positioned over the left adrenalgland, then the system can perform localization to the left adrenalgland. 403 illustrates a processing block of displaying at least one ofthe group consisting of: a digital object (varying size, shape, color,symbols, texture, etc.) at the corresponding first coordinate in animage of the second 3D dataset; and, an imaging slice of the second 3Ddataset containing the sub-structure.

FIG. 5 illustrates techniques to determine the corresponding firstcoordinate. A first technique is by performing segmentation to determinea structure at the site of a cursor. If the localizer is inside theboundary of the left adrenal gland, then the smart localizer systemwould show a point inside the left adrenal gland in another image. Asecond technique is to use of landmarks, registration points,pseudoregistration points. Examples of landmarks would include a singlecoarse calcification within the spleen, which could readily berecognized. An example of registration point would include the centerpoint of an organ. An example of a pseudoregistration point would be apoint that has a defined distance between two registration points. Athird technique is to use of an organ-specific coordinate system. Anorgan-based coordinate system can be established to improve localizationby defining precise coordinates within an organ. For example, aspherical coordinate system can be performed for the liver wherein theorigin is defined as the center point. A fourth technique is to performan artificial intelligence algorithm, which utilizes training datacomprises sets of longitudinal 3D imaging examinations with embeddedlocalization points (e.g., an example would be a CT dataset from 2019with a point on a left adrenal gland and a CT dataset from 2018 with acorresponding point on the left adrenal gland). A key point of noveltyis the use of point and corresponding point as part of the trainingdataset. Techniques include those discussed in U.S. patent applicationSer. No. 16/939,192, Radiologist Assisted Machine Learning, which isincorporated by reference in its entirety.

FIG. 6A illustrates a first example of the smart localization system. APACS display is shown with a first CT scan of the abdomen from 2019 onthe left and a second CT scan of the abdomen from 2018 on the right. 600illustrates a cursor positioned on the 2019 image at a firstsub-structure location (posterolateral limb) on the structure (adrenalgland). 600 illustrates a digital object positioned on the 2018 image atthe corresponding first sub-structure location (posterolateral limb) onthe structure (adrenal gland).

FIG. 6B illustrates a first example of the smart localization systemwherein the first coordinate is determined by a fixation location. APACS display is shown with a first CT scan of the abdomen from 2019 onthe left and a second CT scan of the abdomen from 2018 on the right. 602illustrates the site of a fixation location positioned on the 2019 imageat a first sub-structure location (posterolateral limb) on the structure(adrenal gland). 603 illustrates a digital object positioned on the 2018image at the corresponding first sub-structure location (posterolaterallimb) on the structure (adrenal gland). As soon as the user looks overto the second image, the eye tracking system displays the digital objecton the second screen at a location corresponding to the last fixationlocation on the first screen. Thus, both cursors and eye trackingsystems can determine the location of the first coordinate. The eyetracking system is discussed further in U.S. patent application Ser. No.16/842,631, A SMART SCROLLING SYSTEM, which is incorporated by referencein its entirety.

FIG. 7 illustrates display options for the corresponding firstcoordinate. A table is shown with display options for the digital objectfor the corresponding first coordinate. Regarding the appearance of thedigital object at the corresponding first coordinate, the appearanceincludes: variable color/grayscale (e.g., red, blue, white, yellow);variable size (e.g., 0.5 mm, 1 cm); variable shape (e.g., round, star);and, other variations (e.g., blinking, solid, etc.). Regarding thetiming of display of the digital object at the corresponding firstcoordinate, the appearance includes: displaying the digital object atall times; and, displaying the digital object only when user is lookingat the monitor displaying the first coordinate (which is advantageousbecause the user would not see a mobile object in the periphery of thevisual field).

FIG. 8 illustrates improvements, which are enabled by implementing anorgan-specific coordinate system. First, accurate voxel-by-voxelanalysis is enabled. The current system mitigates errors in registrationbecause a voxel at a first coordinate on a first exam can be comparedwith a voxel at a corresponding first coordinate on a second exam. Thisenables a comparative analysis of interval change between voxel and/orcluster of voxels at the first imaging examination with voxel and/orcluster of voxels at the second examination (e.g., generatingorgan-specific coordinate system so that each voxel in the segmentedorgan has a specific coordinate that is reproducible within the organ).Second, includes an enhanced accuracy during radiation treatment. Anorgan-specific coordinate system can be established. It is anticipatedthat various researchers and professional societies will establishpreferred coordinate systems for each organ. For example, the AmericanCollege of Radiology can establish a preferred coordinate system for theliver. Consider a first example. The liver could be segmented into thetraditional 8 segments. The caudate lobe of the liver can have its owncoordinate system with the origin at the center of the caudate lobe anda spherical coordinate system can be used. Alternatively, the entireliver could have its own coordinate system (e.g., spherical). Suchorgan-specific coordinate systems would enable more precise localizationof small lesions (e.g., 1 cm) within the liver. Third, includes enhancedaccuracy during surgery. A range of technologies are emerging in thefield of surgery. For example, augmented reality systems are workingtheir way into the operating room. In some embodiments, the augmentedreality headset can be geo-registered with the organ-specific coordinatesystem, which will enable precision localization of a small lesion(e.g., 1 cm) within the liver.

FIG. 9A illustrates identification of internal reference points at aninitial examination and two spots where the user wants to localize to ona subsequent examination. 900 illustrates a first reference point. 902illustrates a second reference point. 904 illustrates a third referencepoint. 906 illustrates a first area where localization to a subsequentexamination is desired to occur. Note that the first localization spot906 is in between the second reference point 902 and the third referencepoint 904. 908 illustrates a second area where localization to asubsequent examination is desired to occur.

FIG. 9B illustrates identification of internal reference points at asubsequent examination with two spots where the user has localized to ascompared to the initial examination. 900 illustrates a first referencepoint. 902 illustrates a second reference point. 904 illustrates a thirdreference point. 906 illustrates a first localization spot to the priorexamination (906 as shown in FIG. 9A). Note that the first localizationspot 906 on FIG. 9B subsequent examination is in between the secondreference point 902 and the third reference point 904 and also note thatthis is in a very similar position as compared to FIG. 9A, right inbetween the second reference point 902 and the third reference point904. The preferred embodiment would be to place the perform a lineartransform to determine the location of the first localization spot 906.To teach this, assume at the first time point in FIG. 9A that thedistance between the second reference point 902 and the third referencepoint 904 is 4 cm and the first localization spot 906 is clicked at alocation exactly 3 cm from the second localization spot 902 and 1 cmfrom the third localization spot 904 and along the line connecting thesecond localization spot 902 and the third localization spot 904. Also,assume at the second time point in FIG. 2B that the distance between thesecond reference point 902 and the third reference point 904 is now 6cm. If the first localization spot 906 should appear at a locationexactly 4.5 cm from the second localization spot 902 and 1.5 cm from thethird localization spot 904 and along the line connecting the secondlocalization spot 902 and the third localization spot 904. Also, notethat a second localization spot 908 is immediately above the firstlocalization spot 906 and note the position of the first reference point900 in FIG. 9A. Note that the first reference point 900 has shifted tothe left and so too has the second localization spot 908, also moving tothe left. A linear transformation could be performed. Alternatively, anon-linear transformation could also be performed. In essence, thismethod provides registration points and performs transformations forprecision localization between multiple cross-sectional imagingexaminations. Ultimately, this system can also perform precision markupof lesions, enabling precision tracking of lesions and improvedcommunication between radiologists. In some embodiments, theregistration spots represent distinguishable features (e.g., coarsecalcifications) in images. Also, it should be noted that theregistration spots can be used in conjunction with the coordinatesystems, as discussed elsewhere in this patent.

FIG. 10A illustrates the liver and an anatomic structure specificcoordinate system (could also be called “organ specific coordinatesystem”). An example anatomic structure specific coordinate system 1000is illustrated. The liver 1002 is illustrated. The center coordinate1004 of the liver is illustrated at (0,0,0). All voxels within theimaging volume are assigned a coordinate within the anatomic structurespecific coordinate system. This is important because a radiologist orsurgeon could give a precise location. Current example descriptors forlocalizing a tumor include: “tumor is located at the upper pole of thekidney”; “tumor is located in the right lower thyroid lobe”; “tumor islocated in the dome of the liver”. This patent provides a precisionmethod to describe location. For example, assume that the center of theliver is located at (0,0,0) and the language would in the preferredembodiment of this patent states “liver tumor coordinate of (4.4 cm, 90degrees, 70 degrees).” This precision description is supplemented by thefact that subsequent examinations could physically load an annotation atthis coordinate to provide a more comprehensive examination. This isspecifically important because sometimes a tumor disappears after afirst set of chemotherapy is delivered. And, it isn't until manyexaminations later when it comes back. There could have been 20 or moresites of tumor on an exam from years ago, but on the recent examinationsonly 5 sites of tumor. So, the radiologist focuses on looking at the 5sites of tumor and could potentially miss the fact that some of theother sites present on the remote scan from years ago are coming back.This organ-specific coordinate system is much more useful and effectivethan an (x,y,z) coordinate system illustrated in FIG. 1 that theradiologist doesn't even look at nor care about. This coordinate systemis so useful in fact because even if an organ (e.g., liver) moves on asubsequent examination, the organ specific coordinate will remain thesame and the voxels within the organ specific coordinate system willremain the same.

FIG. 10B illustrates a CT image of the liver and example coordinate of avoxel in the liver. 1006 illustrates the origin of the liver. 1008illustrates an arbitrary point within the liver, whose coordinates are(r=7.4 cm, y=90 degrees, 0=0 degrees).

FIG. 11 illustrates the adrenal gland and two different organ specificcoordinate systems. It should be noted that a range of coordinatesystems can be used for the various organs in the body. An example organspecific coordinate system 1100, which is a spherical coordinate systemis illustrated. The adrenal gland 1102 is illustrated. The centercoordinate 1104 of the adrenal gland is illustrated, both within theadrenal gland 1102 and within the organ specific coordinate system 1100.In the preferred embodiment, the coordinate at the center of each organspecific coordinate system would be (0,0,0). Note that a uniquecoordinate system would be performed for each organ (e.g., liver has itsliver specific coordinate system with center at (0,0,0), pancreas hasits pancreas specific coordinate system with center at (0,0,0), etc.).This is the most straight forward strategy for an organ specificcoordinate system would be to implement a conventional coordinatesystem, such as Cartesian coordinate system; Cylindrical coordinatesystem; Polar coordinate system; and, Spherical Coordinate system. Oneof the limitations of a spherical coordinate system, however, is thefact that for an organ like the adrenal gland which has “limbs” or“arms”, is the fact that portions of it could be bent or shifted inposition between scans. To overcome this, more complex mathematicalmodels can and should be implemented. Specifically, the adrenal gland“fingers” commonly curve over distance from the center and the amount ofcurvature is variable from scan to scan. This can be related to theamount of mass effect on the adrenal gland from other adjacent organs,or possibly patient positioning. An example mathematical model of theadrenal gland can be performed by plotting a centerline 1106 along eachlimb. The superior limb 1108 of the adrenal gland is illustrated. Theneach voxel could be plotted as distance along the center line 1106 plusorthogonal distance and direction away from the center line 1106. Forexample, voxel 1110 and orthogonal distance 1112 is shown. 1114 is anexample of a course calcification, which can be used as a referencepoint. This coordinate system could also be applied to tumors forprecision analysis. For distinctly bilobed structures (e.g., a tumorshaped as two balls connected by a thin band, a dual coordinate systemcould be established. The purpose of this note that each segmentedstructure should have its own unique organ specific coordinate that canbe tracked over time to determine interval change. Since some tissuesare quite deformable, internal ratios need to be established with thismodel (e.g., such as the making nipple a 0 and the chest wall a 1 and ifthe lesion is located at 90% the way back it is given a 0.9 ratio). So,a coordinate system for this would be (r,Θ,φ). Such a system wouldinternally correct for amount of deformation (flattening vs. elongating)of the breast tissue. This overall process serves to provide pinpointprecision coordinates for a specific spot in an anatomic feature.Ultimately, the entire body will be mapped with pinpoint voxel-by-voxelaccuracy.

FIG. 12 illustrates reference points, pseudoreference points and ensuinganalysis. 1200 illustrates a first reference point. 1202 illustrates asecond reference point. 1204 illustrates a third reference point. 1206illustrates a pseudoreference point. Reference points, as defined inthis patent specification, are anatomic features that can be mapped overmultiple examinations. A pseudoreference point is defined as a locationwhich does not have any specific anatomic feature that is easilyrecognizable but can be used to assist with analysis. For example,measurements and voxel analysis can be performed along the distancesfrom any pair of reference points, such as in between the firstreference point 1200 and the second reference point 1202. The analysiscan be in a linear fashion, such as shown in 1208. Alternatively, theanalysis can be performed in a curvilinear fashion, such as is shown in1210. Note that curvilinear fashion analysis would need to have anorientation and direction. The analysis that needs to be performed isthe length, single or multiple voxel analysis, such as examining dataunits (e.g., Hounsfield Units) and other trajectories. Additionally,analysis can be performed between two pseudoreference points, apseudoreference point and a reference point, such as is shown by 1208 inbetween the first reference point 1200 and the pseudoreference point1206. This patent provides a method therefore of also using artificiallycreated pseudoreference points when there are no reliable, reproducibleanatomic features readily available for analysis. This process can beused with U.S. patent application Ser. No. 16/195,251 Interactive voxelmanipulation strategies in volumetric medical imaging enables virtualmotion, deformable tissue, and virtual radiological dissectionconnective tissue properties, which is incorporated by reference in itsentirety. For example, certain connective tissues can be modeled inassociation with both reference points and pseudoreference points.Another embodiment will be to perform precision surgical guidance andradiology oncology treatment.

FIG. 13 illustrates the plotting of organ specific voxel coordinates andtheir associated data units. Note that an anomaly is seen on time point#3 for the organ specific voxel coordinate located at (0.1 mm, 0,0).This is an illustrative chart showing that voxels can be analyzed bytheir organ specific coordinate system.

FIG. 14A illustrates a specific lesion in an organ. 1400 illustrates a 5cm distance scale. 1401 illustrates a kidney of a patient at 5 yearsold. Note that the kidney 1401 is approximately 5 cm in maximumdimension. 1402 illustrates a lesion within the kidney. A coordinatesystem can be established, such as (“percentage from origin to peripheryof organ”, φ, Θ). For instance, lesion 1402 would be located atcoordinate (50%, 0 degrees, 90 degrees). Additionally, data on thedistance could also be included (50%, 2.5 cm, 0 degrees, 90 degrees).

FIG. 14B illustrates how a coordinate system can track a lesion in agrowing organ. Note that the kidney 1403 is approximately 10 cm inmaximum dimension, in this patient who is now 15 years old. The questionis asked “did the lesion at 1402 change in size?” To answer this, it isfirst necessary to track the location of the lesion. Ideally, acoordinate system would be able to find it even though the patient hasgrown from age 5 to age 15. The coordinate system established in FIG.14A illustrates that lesion 1402 is still at coordinate (50%, 0 degrees,90 degrees).

FIG. 15A illustrates a specific lesion in an organ, wherein the organ isin a first orientation. 1500 illustrates vertebral bodies, which areused as an external reference point for orientation purposes. 1501illustrates a kidney of a patient in 2018. 1502 illustrates a lesionwithin the kidney. A coordinate system can be established, such as(“percentage from origin to periphery of organ”, φ, θ). For instance,lesion 1502 would be located at coordinate (50%, 0 degrees, 90 degrees).

FIG. 15B illustrates how an organ specific coordinate system can track alesion in a subsequent scan wherein the organ has a differentorientation. 1500 illustrates vertebral bodies, which are used as anexternal reference point for orientation purposes. 1503 illustrates thekidney in a new orientation as compared to the vertebral bodies in a2019 scan. In using the organ-specific coordinate system, the lesion1502 would be still located at coordinate (50%, 0 degrees, 90 degrees),which provides improved tracking as compared to the coordinate systemused in FIG. 1.

FIG. 16A illustrates a specific lesion in an organ, wherein the organ isin a first configuration. 1600 illustrates a 5 cm distance scale. 1601illustrates a kidney of a patient in 2018, wherein the kidney 1601 has afirst morphologic configuration. 1602 illustrates a lesion within thekidney. A coordinate system can be established, such as (“percentagefrom origin to periphery of organ”, φ, Θ). For instance, lesion 1602would be located at coordinate (50%, 0 degrees, 90 degrees). Voxelationof the kidney can be performed so that voxels contain similar tissueover multiple examinations. For example, a chunk (or portion or section)of tissue can be assigned to voxel 1603, which is isotropic.

FIG. 16B illustrates how an organ specific coordinate system can track alesion in a subsequent scan wherein the organ has a differentconfiguration. 1600 illustrates a 5 cm distance scale. 1603 illustratesthe kidney in a new configuration as compared to the vertebral bodies ina 2019 scan. The new configuration could be due to conditions such asexternal mass effect on the kidney 1603. In using the organ-specificcoordinate system, the lesion 1602 would be still located at coordinate(50%, 0 degrees, 90 degrees), which provides improved tracking ascompared to the coordinate system used in FIG. 1. Voxelation of thekidney can be performed so that voxels contain similar tissue overmultiple examinations. For example, the same chunk (or portion orsection) of tissue assigned to isotropic voxel 1603 can be assigned tovoxel 1604, which is anisotropic. This allows comparison of similarsections of tissue while the morphology of the organ changes. Such aprocess improves conventional voxel-based morphometry (e.g., inneuroimaging) because it accounts for changes in configuration of organsin between examinations.

FIG. 17 illustrates multiple coordinate systems in a single structure.1700 illustrates a structure (e.g., tumor). 1701 illustrates an anatomicstructure specific coordinate system (e.g., coordinate system describedin FIG. 14A and FIG. 14B). 1702 illustrates an origin of the coordinatesystem 1701. 1703 illustrates a sub-structure within the tumor, such asa solid nodular component. 1704 illustrates a coordinate system of thesub-structure. 1705 illustrates an origin of the coordinate system ofthe sub-structure 1704. 1706 illustrates a voxel pertaining to thestructure 1700. 1707 illustrates a voxel pertaining to the sub-structure1703. 1708 illustrates a distance from the origin of the structure 1702to the origin of the sub-structure 1705. To describe a coordinate systemwithin the nodule, a first coordinate system of the structure canreference a second coordinate system of the sub-structure.

FIG. 18 illustrates the use of an anatomic structure-specific coordinatesystem for each segmented structure in an imaging examination. 1800illustrates a first vertebral body, which has an organ-specificcoordinate system. 1801 illustrates a second vertebral body, which hasan organ-specific coordinate system. 1802 illustrates a third vertebralbody, which has an organ-specific coordinate system. 1803 illustrates afourth vertebral body, which has an organ-specific coordinate system.1804 illustrates a fifth vertebral body, which has an organ-specificcoordinate system. 1805 illustrates a sixth vertebral body, which has anorgan-specific coordinate system. 1806 illustrates a liver, which has anorgan-specific coordinate system. 1807 illustrates a right kidney, whichhas an organ-specific coordinate system. 1808 illustrates a left kidney,which has an organ-specific coordinate system. In some embodiments,precision inter-organ relationships can be performed. For example, themajor axis of the kidney (as defined as the axis from the superior poleto the inferior pole can be performed for each kidney. A longitudinalanalysis of the relationship between the right kidney and the leftkidney can be performed over multiple examinations. For example, on afirst time point, the major axis of the right kidney could be directedsuperior-inferior and the major axis of the left kidney could bedirected in a parallel fashion (as compared to the major axis of theright kidney). On a second time point, the major axis of the rightkidney could be directed superior-inferior and the major axis of theleft kidney could be directed in an oblique fashion (as compared to themajor axis of the right kidney). A new change in orientation couldindicated pathology, such as mass effect from a growing tumor. Theassigning of a structure-specific coordinate system could also be usefulin a range of applications. For example, it would be able to answer avariety of questions, such as are the kidneys being pushed apart overtime (e.g., by a tumor). For example, a distance from the origin of theright kidney 1807 to the left kidney 1808 is illustrated as 1809. Alarge amount of inter-organ relationships can be assessed bymeasurements. It could also help determine changes in scoliosis byprecisely mapping what is happening to each vertebrae and can refinetreatments. It is important to note that different segmented structurescan have different organ-specific coordinate systems. For example, therigid vertebrae can have a cartesian coordinate system. The deformablekidney could have a spherical coordinate system. The adrenal gland canbe modeled with a custom coordinate system, as previously discussed.

FIG. 19A illustrates a single coordinate system for the liver. The liverand a single coordinate system are shown. This would be the simplestmethod and is good in most situations. However, in some situations wherea portion of the liver is deformed on an imaging examination (e.g., masseffect), errors could occur. In these situations, multiple coordinatesystems can be used and can overcome these type errors, such as taughtin FIG. 19B.

FIG. 19B illustrates a multiple coordinate systems for the liver. Notethat this illustration shows that the liver has been segmented intoeight segments. Each segment has its own coordinate system. This is amore robust system and can provide accurate tracking of liver lesionslongitudinally even in the setting of post-surgical changes or otherdeformities to the liver that might occur.

Throughout the entirety of the present disclosure, use of the articles“a” or “an’ to modify a noun may be understood to be used forconvenience and to include one, or more than one of the modified noun,unless otherwise specifically stated. Elements, components, modules,and/or parts thereof that are described and/or otherwise portrayedthrough the figures to communicate with, be associated with, and/or bebased on, something else, may be understood to so communicate, beassociated with, and or be based on in a direct and/or indirect manner,unless otherwise stipulated herein. The device(s) or computer systemsthat integrate with the processor(s) may include, for example, apersonal computer(s), workstation(s) (e.g., Sun, HP), personal digitalassistant(s) (PDA(s)), handheld device(s) such as cellular telephone(s),laptop(s), handheld computer(s), or other device(s) capable of beingintegrated with a processor(s) that may operate as provided herein.Accordingly, the devices provided herein are not exhaustive and areprovided for illustration and not limitation. References to “amicroprocessor and “a processor, or “the microprocessor and “theprocessor.” may be understood to include one or more microprocessorsthat may communicate in a stand-alone and/or a distributedenvironment(s) and may thus be configured to communicate via wired orwireless communications with other processors, where such one or moreprocessor may be configured to operate on one or moreprocessor-controlled devices that may be similar or different devices.Use of such “microprocessor or “processor terminology may thus also beunderstood to include a central processing unit, an arithmetic logicunit, an application-specific integrated circuit (IC), and/or a taskengine, with such examples provided for illustration and not limitation.Furthermore, references to memory, unless otherwise specified, mayinclude one or more processor-readable and accessible memory elementsand/or components that may be internal to the processor-controlleddevice, external to the processor-controlled device, and/or may beaccessed via a wired or wireless network using a variety ofcommunications protocols, and unless otherwise specified, may bearranged to include a combination of external and internal memorydevices, where Such memory may be contiguous and/or partitioned based onthe application.

Accordingly, references to a database may be understood to include oneor more memory associations, where such references may includecommercially available database products (e.g., SQL, Informix, Oracle)and also include proprietary databases, and may also include otherstructures for associating memory Such as links, queues, graphs, trees,with such structures provided for illustration and not limitation.References to a network, unless provided otherwise, may include one ormore intranets and/or the Internet, as well as a virtual network.References hereinto microprocessor instructions ormicroprocessor-executable instructions, in accordance with the above,may be understood to include programmable hardware.

Unless otherwise stated, use of the word “substantially’ may beconstrued to include a precise relationship, condition, arrangement,orientation, and/or other characteristic, and deviations thereof asunderstood by one of ordinary skill in the art, to the extent that suchdeviations do not materially affect the disclosed methods and systems.Throughout the entirety of the present disclosure, use of the articles“a” or “an’ to modify a noun may be understood to be used forconvenience and to include one, or more than one of the modified noun,unless otherwise specifically stated. Elements, components, modules,and/or parts thereof that are described and/or otherwise portrayedthrough the figures to communicate with, be associated with, and/or bebased on, something else, may be understood to so communicate, beassociated with, and or be based on in a direct and/or indirect manner,unless otherwise stipulated herein. Although the methods and systemshave been described relative to a specific embodiment thereof, they arenot so limited. Obviously, many modifications and variations may becomeapparent in light of the above teachings. Many additional changes in thedetails, materials, and arrangement of parts, herein described andillustrated, may be made by those skilled in the art. Having describedpreferred embodiments of the invention it will now become apparent tothose of ordinary skill in the art that other embodiments incorporatingthese concepts may be used. Additionally, the software included as partof the invention may be embodied in a computer program product thatincludes a computer useable medium. For example, such a computer usablemedium can include a readable memory device, such as a hard drivedevice, a CD-ROM, a DVD ROM, or a computer diskette, having computerreadable program code segments stored thereon. The computer readablemedium can also include a communications link, either optical, wired, orwireless, having program code segments carried thereon as digital oranalog signals. Accordingly, it is submitted that that the inventionshould not be limited to the described embodiments but rather should belimited only by the spirit and scope of the appended claims.

Several features, aspects, embodiments and implementations have beendescribed. Nevertheless, it will be understood that a wide variety ofmodifications and combinations may be made without departing from thescope of the inventive concepts described herein. Accordingly, thosemodifications and combinations are within the scope of the followingclaims.

The invention claimed is:
 1. A method comprising: loading a first 3Dimaging dataset into an image processing workstation wherein the first3D dataset comprises a voxelated dataset of a scanned volume at a firsttime point; loading a second 3D imaging dataset into the imageprocessing workstation wherein the second 3D dataset comprises avoxelated dataset of the scanned volume at a second time point;performing a smart localization system comprising: determining a firstcoordinate of an image of a first 3D dataset wherein the firstcoordinate is enclosed within a structure wherein the first coordinateis located at a sub-structure location by at least one of the groupconsisting of: positioning a cursor; and utilizing an eye trackingsystem; determining a corresponding first coordinate in a second 3Ddataset, wherein the corresponding first coordinate is enclosed withinthe structure and wherein the corresponding first coordinate is locatedat the sub-structure location; and displaying at least one of the groupconsisting of: a digital object at the corresponding first coordinate inan image of the second 3D dataset; and an imaging slice of the second 3Ddataset containing the sub-structure.
 2. The method of claim 1 furthercomprising utilizing in the smart localization system at least onereference point within the structure comprising wherein the at least onereference point is selected from the group consisting of: a centerpoint; a superior most point; an inferior most point; a medial mostpoint; a lateral most point; an anterior most point; a posterior mostpoint; and a recognizable anatomic feature.
 3. The method of claim 2further comprises wherein the at least one reference point is used forat least one of the group consisting of: volumetric analysis; and,morphologic analysis.
 4. The method of claim 2 further comprisingutilizing at least one pseudoreference point within the structurecomprising wherein the at least one pseudoreference point is a point ata distance in between at least two reference points.
 5. The method ofclaim 4 further comprises wherein the at least one pseudoreference pointis used for at least one of the group consisting of: volumetricanalysis; and, morphologic analysis.
 6. The method of claim 2 furthercomprising utilizing in the smart localization system a coordinatesystem based comprising at least one of the group consisting of: acartesian coordinate system; a cylindrical coordinate system; a polarcoordinate system; a spherical coordinate system; and an organ specificcoordinate system.
 7. The method of claim 6 further comprises assigninga precision location of a lesion wherein the precision locationcomprises a coordinate location on the coordinate system.
 8. The methodof claim 6 further comprising inputting the precision location of thelesion in a radiology report.
 9. The method of claim 8 further comprisesgenerating at least two coordinate systems for the structure.
 10. Themethod of claim 8 further comprises using the coordinate system for atleast one of the group consisting of: a radiation treatment; and asurgical procedure.
 11. The method of claim 6 further comprisesinputting an annotation at the site of the precision location of thelesion on an image.
 12. The method of claim 1 further comprising whereinwhen the structure changes in size from the first 3D dataset to thesecond 3D dataset, the determining of the corresponding first coordinatein the second 3D dataset accounts for the structure's changes in size.13. The method of claim 1 further comprising wherein when the structurechanges in configuration from the first 3D dataset to the second 3Ddataset, the determining of the corresponding first coordinate in thesecond 3D dataset accounts for the structure's changes in configuration.14. The method of claim 1 further comprising wherein when the structurechanges in orientation from the first 3D dataset to the second 3Ddataset, the determining of the corresponding first coordinate in thesecond 3D dataset accounts for the structure's changes in orientation.15. The method of claim 1 further comprising: determining a referenceaxis for the volume in the first 3D dataset; determining an axis of thestructure in the first 3D dataset; determining a first angle wherein thefirst angle is an angle between the reference axis for the volume in thefirst 3D dataset and the axis of the structure in the first 3D dataset;determining a corresponding reference axis for the volume in the second3D dataset; determining a corresponding axis of the structure in thesecond 3D dataset; determining a second angle wherein the second angleis an angle between the corresponding reference axis for the volume inthe second 3D dataset and the corresponding axis of the structure in thesecond 3D dataset; and comparing the first angle with the second angleto determine an interval change.
 16. The method of claim 1 furthercomprising performing an analysis of interval change between a voxel atthe sub-structure in the first 3D dataset and a voxel at thesub-structure in the second 3D dataset.
 17. The method of claim 1further comprises determining the corresponding first coordinate byutilizing an artificial intelligence system, wherein the artificialintelligence system utilizes training data comprises sets oflongitudinal 3D imaging examinations with embedded localization points.18. A non-transitory computer readable medium having computer readablecode thereon for image processing, the medium comprising: performing asmart localization system comprising: determining a first coordinate ofan image of a first 3D dataset wherein the first coordinate is enclosedwithin a structure wherein the first coordinate is located at asub-structure location by at least one of the group consisting of:positioning a cursor; and utilizing an eye tracking system; determininga corresponding first coordinate in a second 3D dataset, wherein thesecond 3D dataset contains the structure, wherein the correspondingfirst coordinate is enclosed within the structure and wherein thecorresponding first coordinate is located at the sub-structure location;and displaying at least one of the group consisting of: a digital objectat the corresponding first coordinate in an image of the second 3Ddataset; and an imaging slice of the second 3D dataset containing thesub-structure.
 19. An apparatus comprising: an IO device; and an imageprocessor in communication with the IO device, the image processorscomprising a program stored on a computer-readable non-transitory media,the program comprising instructions that: perform a smart localizationsystem comprising: determining a first coordinate of an image of a first3D dataset wherein the first coordinate is enclosed within a structurewherein the first coordinate is located at a sub-structure location byat least one of the group consisting of: positioning a cursor; andutilizing an eye tracking system; determining a corresponding firstcoordinate in a second 3D dataset, wherein the second 3D datasetcontains the structure, wherein the corresponding first coordinate isenclosed within the structure and wherein the corresponding firstcoordinate is located at the sub-structure location; and display atleast one of the group consisting of: a digital object at thecorresponding first coordinate in an image of the second 3D dataset; andan imaging slice of the second 3D dataset containing the sub-structure.